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DETAILED ACTION 

Response to Amendment 

1. This office action is in response to an amendment/response filed on August 6 th , 2010. 

2. Claims 1, 9, 17, 26, 35, 40, 45, 50, and 55 have been amended. 

3. No claims have been cancelled. 

4. No claims have been added. 

5. Claims 1-3, 7-11, 15-19, 23-28, 32-57 and 59 are currently pending. 

Claim Rejections - 35 USC § 103 

6. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 1 02 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

7. The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 
(1966), that are applied for establishing a background for determining obviousness under 35 
U.S.C. 103(a) are summarized as follows: 

1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating obviousness 
or nonobviousness. 

8. This application currently names joint inventors. In considering patentability of the 
claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of the various 
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claims was commonly owned at the time any inventions covered therein were made absent any 
evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out 
the inventor and invention dates of each claim that was not commonly owned at the time a later 
invention was made in order for the examiner to consider the applicability of 35 U.S.C. 103(c) 
and potential 35 U.S.C. 102(e), (f) or (g) prior art under 35 U.S.C. 103(a). 

9. Claim 1-3, 7-11, 15-19, 23-28, 32-57 and 59 rejected under 35 U.S.C. 103(a) as being 
unpatentable over Fernandez et al. (U.S. Patent 6,922,664) in view of Kumar et al. (U.S. Patent 
7,188,151) and further in view of Stanger et al. (U.S. Patent 7,376,159). 

For claim 1, Fernandez et al. disclose an information transmission method, comprising: 

acquiring one or more of audio information and video information of a performance at a 
given location (see figure 2 and column 2 lines 23-34, which recite a microphone 44 and camera 
42 for acquiring audio and visual information); 

detecting, concurrent with the acquiring of the one or more of audio information and 
video information, bio-information of at least one individual participator present at the given 
location (see figure 2,figure 4 and column 2 lines 23-34, which recite a biometric sensor array 
module 50 that detects bio-information); 

packetizing the acquired one or more of audio information and video information into a 
stream of first data packets (see column 1 lines 62-65 and figure 2, which recite using TCP/IP 
packets to transmit audio and video information gathered by microphone 44 and camera 42 from 
client 20 to network 10); 
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performing statistical processing of the bio-information (see column 3 lines 48-67 and 
column 4 lines 1-19, which recite statistical processing of the bio-information by collecting and 
interpreting the mix-mode signals of the bio sensors 50 into findings, flags, warnings, or other 
indications provided as feedback); 

packetizing the statistically processed bio-information (see column 4 lines 8-19, which 
recite generating digital packets comprising the collected and interpreted bio-information 
signals collected by biometric sensor array module 50); 

transmitting the multiplexed stream (see figure 5b, column 4 lines 8-19, and column 7 
lines 1-9, which recites a communication module 94 that transmits audio/video signals as well as 
signals from the bio-information sensors in real-time). 

Fernandez et al. disclose all the subject of the claimed invention with the exception 
wherein the statistical processing of the bio-information is specifically performed over 
predetermined time intervals corresponding to respective pluralities of the first data packets of 
the one or more of audio information and video information, wherein the packetizing of 
statistically processed bio-information is specifically packetized into respective pluralities of 
second data packets corresponding to the respective pluralities of the first data packets. 

However, Kumar et al. disclose a system and method for real-time retrieval, assessment, 
analysis, and transmission of physiological data such as EKG, EMG, EEG, blood pressure, body 
temperature, and pulse measurements (see column 5 lines 47-61 and column 6 lines 50-56). The 
real-time streaming of such bio-information is augmented by transmission of audio and video 
streams that are synchronized with the respective bio-information (see column 7 lines 4-14). 
Thus, it would have been obvious to the person of ordinary skill in the art at the time of the 
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invention to use the system and method for communicating bio-information and respectively 
synchronized audio and video information as taught by Kumar et al. with the method for 
transmitting and receiving bio-information with audio and video information as taught by 
Fernandez et al. The packetized bio-information must be multiplexed and transmitted such that 
it is temporally adjacent to the audio and video packets to ensure that the bio-information is 
synchronized with the audio and video information to form a real-time combined data stream. If 
the bio-information and audio/video information of a corresponding time period are not 
multiplexed together, then one of the data information would arrive later. As a result, the final 
stream containing video, audio, and bio-information would be delayed until the late data arrives. 
In this case, the delay would prevent the real-time streaming of the combined data streams as 
disclosed by Kumar et al. Thus, the packetized bio-information must be multiplexed together 
with the audio and video packets to ensure that the bio-information is synchronized with the 
audio and video information to form a real-time combined data stream as disclosed by Kumar et 
al. The motivation for using the system and method for transmitting bio-information and 
respectively synchronized audio and video information as taught by Kumar et al. with the 
method for transmitting bio-information with audio and video information as taught by 
Fernandez et al. is to enhance the communication experienced by users using transmission of 
audio/video information with respectively synchronized bio-information. 

Fernandez et al. and Kumar et al. disclose all the subject of the claimed invention with 
the exception of specifically partitioning the acquired one or more of audio information and 
video information into lengths of partitioned data each of which corresponds to a predefined time 
interval, compressing and packetizing the lengths of partitioned data into a stream of first data 
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packets having empty regions resulting from the compression of lengths of the partitioned data, 
and multiplexing the stream of first data packets with the second data packets by inserting a 
respective plurality of second data packets, which corresponds to one of the predetermined time 
intervals, into one of the empty regions of the stream that is adjacent to a respective plurality of 
first data packets which corresponds to a same one of the predetermined time intervals. 

However, Stanger et al. from the same or similar fields of endeavor disclose a system and 
method for adding auxiliary data to empty space that results from compressing a data stream for 
transmission (see column 2 lines 7-37). Specifically, the system and method partitions video and 
audio data into data packets (see column 5 lines 10-27 and figures 3a-3b), compresses the 
packets to provide empty regions (see column 5 lines 31-65 and figure 4), and inserts auxiliary 
data in the empty spaces (see column 7 lines 41-65 and figure 5). Thus, it would have been 
obvious to the person of ordinary skill in the art at the time of the invention to use the system and 
method for adding auxiliary data to empty space that is the result of compressing a data stream as 
taught by Stanger et al. with the method for transmitting and receiving synchronized bio- 
information with audio and video information as taught by Fernandez et al. and Kumar et al. 

As noted above in reference to Kumar et al., the bio-information is synchronized with 
respective audio and video information to form a real-time data stream . Thus, bio-information 
must be multiplexed with audio/video information that all correspond to the same time period. If 
the respective bio-information is not multiplexed with audio/video information that all 
correspond to the same time period, then one of the data information would arrive later. As a 
result, the final stream containing video, audio, and bio-information would be delayed until the 
late data arrives. In this case, the delay would prevent the real-time streaming of the combined 



Application/Control Number: 10/541,278 Page 7 

Art Unit: 2464 

data streams as disclosed by Kumar et al. Thus, the packetized bio-information of a time period 
must be multiplexed in empty regions that correspond to audio and video packets of the same 
time period to ensure that the bio-information is synchronized with the audio and video 
information to form a real-time combined data stream as disclosed by Kumar et al. The system 
and method for adding auxiliary data to empty space in a data stream can be implemented by 
inserting the synchronized bio-information as taught by Fernandez et al. and Kumar et al. to the 
empty space that results from compressing the audio and video data using the encoders and 
multiplexers as taught by Stanger et al. The motivation for using the system and method for 
adding auxiliary data to empty space that results from compressing a data stream as taught by 
Stanger et al. with the method for transmitting and receiving synchronized bio-information with 
audio and video information as taught by Fernandez et al. and Kumar et al. is to improve the 
efficiency of the system by creating more payload capacity in the existing transmission channels. 

For claim 2, Fernandez et al. discloses an information transmission method, wherein the 
at least one individual includes a speaker, a player, an actor, an actress, or a conductor who 
serves as a source of the audio information, and/or a performer or a photographed person who is 
included within the video information (see column 7 lines 1-9, which recite real-time conference 
that transmits information of the speaker including bio-information gathered from multi-sensor 
array 50). 

For claim 3, Fernandez et al. discloses an information transmission method, wherein the 
at least one individual includes a listener who is present at the given location place when the 
audio information is acquired and/or a viewer present at the given location when the video 
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information is acquired (see column 7 lines 1-9, which recite real-time conference that transmits 
information of the listener including bio-information gathered from multi-sensor array 50). 

For claim 7, Fernandez et al. discloses an information transmission method, wherein the 
bio-information is selected from the group consisting of body motion, myoelectricity, body 
surface temperature, skin sweating, skin pressure, pulse, breath, micro-vibration, 
cardioelectricity, heartbeat, and blood pressure (see column 4 lines 20-29, which recite detecting 
temperature, pressure, physiological vital information such as pulse and blood pressure using 
the multi-sensor module 50). 

For claim 8, Fernandez et al. discloses an information transmission method, wherein the 
detected bio-information is extracted from the one or more of audio information and video 
information (see column 6 lines 22-25, which recite extracting facial imaging biometric 
information from video information gathered at client 20). 

For claim 9, Fernandez et al. disclose an information transmission apparatus, 
comprising: 

information acquiring means for acquiring one or more of audio information and video 
information of a performance at a given location (see figure 2 and column 2 lines 23-34, which 
recite a microphone 44 and camera 42 for acquiring audio and visual information); 

bio-information detecting means for detecting, concurrent with the acquiring of the one 
or more of audio information and video information, bio-information of at least one individual 
present at the given location (see figure 2,figure 4 and column 2 lines 23-34, which recite a 
biometric sensor array module 50 that detects bio-information); 
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first packetizing means for packetizing the acquired one or more of audio information 
and video information into a stream of first data packets (see column 1 lines 62-65 and figure 2, 
which recite using TCP/IP packets to transmit audio and video information gathered by 
microphone 44 and camera 42 from client 20 to network 10); 

bio-information analysis means for performing statistical processing of the bio- 
information (see column 3 lines 48-67 and column 4 lines 1-19, which recite statistical 
processing of the bio-information by collecting and interpreting the mix-mode signals of the bio 
sensors 50 into findings, flags, warnings, or other indications provided as feedback); 

second packetizing means for packetizing the statistically processed bio-information (see 
column 4 lines 8-19, which recite generating digital packets comprising the collected and 
interpreted bio-information signals collected by biometric sensor array module 50); 

transmission means for transmitting the multiplexed stream (see figure 5b, column 4 lines 
8-19, and column 7 lines 1-9, which recites a communication module 94 that transmits 
audio/video signals as well as signals from the bio-information sensors in real-time). 

Fernandez et al. disclose all the subject of the claimed invention with the exception 
wherein the statistical processing of the bio-information is specifically performed over 
predetermined time intervals corresponding to respective pluralities of the first data packets of 
the one or more of audio information and video information, wherein the packetizing of 
statistically processed bio-information is specifically packetized into respective pluralities of 
second data packets corresponding to the respective pluralities of the first data packets. 

However, Kumar et al. disclose a system and method for real-time retrieval, assessment, 
analysis, and transmission of physiological data such as EKG, EMG, EEG, blood pressure, body 
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temperature, and pulse measurements (see column 5 lines 47-61 and column 6 lines 50-56). The 
real-time streaming of such bio-information is augmented by transmission of audio and video 
streams that are synchronized with the respective bio-information (see column 7 lines 4-14). 
Thus, it would have been obvious to the person of ordinary skill in the art at the time of the 
invention to use the system and method for communicating bio-information and respectively 
synchronized audio and video information as taught by Kumar et al. with the method for 
transmitting and receiving bio-information with audio and video information as taught by 
Fernandez et al. The packetized bio-information must be multiplexed and transmitted such that 
it is temporally adjacent to the audio and video packets to ensure that the bio-information is 
synchronized with the audio and video information to form a real-time combined data stream. If 
the bio-information and audio/video information of a corresponding time period are not 
multiplexed together, then one of the data information would arrive later. As a result, the final 
stream containing video, audio, and bio-information would be delayed until the late data arrives. 
In this case, the delay would prevent the real-time streaming of the combined data streams as 
disclosed by Kumar et al. Thus, the packetized bio-information must be multiplexed together 
with the audio and video packets to ensure that the bio-information is synchronized with the 
audio and video information to form a real-time combined data stream as disclosed by Kumar et 
al. The motivation for using the system and method for transmitting bio-information and 
respectively synchronized audio and video information as taught by Kumar et al. with the 
method for transmitting bio-information with audio and video information as taught by 
Fernandez et al. is to enhance the communication experienced by users using transmission of 
audio/video information with respectively synchronized bio-information. 
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Fernandez et al. and Kumar et al. disclose all the subject of the claimed invention with 
the exception of specifically partitioning the acquired one or more of audio information and 
video information into lengths of partitioned data each of which corresponds to a predefined time 
interval, compressing and packetizing the lengths of partitioned data into a stream of first data 
packets having empty regions resulting from the compression of lengths of the partitioned data, 
and multiplexing the stream of first data packets with the second data packets by inserting a 
respective plurality of second data packets, which corresponds to one of the predetermined time 
intervals, into one of the empty regions of the stream that is adjacent to a respective plurality of 
first data packets which corresponds to a same one of the predetermined time intervals. 

However, Stanger et al. from the same or similar fields of endeavor disclose a system and 
method for adding auxiliary data to empty space that results from compressing a data stream for 
transmission (see column 2 lines 7-37). Specifically, the system and method partitions video and 
audio data into data packets (see column 5 lines 10-27 and figures 3a-3b), compresses the 
packets to provide empty regions (see column 5 lines 31-65 and figure 4), and inserts auxiliary 
data in the empty spaces (see column 7 lines 41-65 and figure 5). Thus, it would have been 
obvious to the person of ordinary skill in the art at the time of the invention to use the system and 
method for adding auxiliary data to empty space that is the result of compressing a data stream as 
taught by Stanger et al. with the method for transmitting and receiving synchronized bio- 
information with audio and video information as taught by Fernandez et al. and Kumar et al. 

As noted above in reference to Kumar et al., the bio-information is synchronized with 
respective audio and video information to form a real-time data stream . Thus, bio-information 
must be multiplexed with audio/video information that all correspond to the same time period. If 
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the respective bio-information is not multiplexed with audio/video information that all 
correspond to the same time period, then one of the data information would arrive later. As a 
result, the final stream containing video, audio, and bio-information would be delayed until the 
late data arrives. In this case, the delay would prevent the real-time streaming of the combined 
data streams as disclosed by Kumar et al. Thus, the packetized bio-information of a time period 
must be multiplexed in empty regions that correspond to audio and video packets of the same 
time period to ensure that the bio-information is synchronized with the audio and video 
information to form a real-time combined data stream as disclosed by Kumar et al. The system 
and method for adding auxiliary data to empty space in a data stream can be implemented by 
inserting the synchronized bio-information as taught by Fernandez et al. and Kumar et al. to the 
empty space that results from compressing the audio and video data using the encoders and 
multiplexers as taught by Stanger et al. The motivation for using the system and method for 
adding auxiliary data to empty space that results from compressing a data stream as taught by 
Stanger et al. with the method for transmitting and receiving synchronized bio-information with 
audio and video information as taught by Fernandez et al. and Kumar et al. is to improve the 
efficiency of the system by creating more payload capacity in the existing transmission channels. 

For claim 10, Fernandez et al. discloses an information transmission information 
acquiring means wherein the at least one individual includes a speaker, a player, an actor, or a 
conductor who serves as a source of the audio information, and/or a performer or a photographed 
person who is included within the video information (see column 7 lines 1-9, which recite real- 
time conference that transmits information of the speaker including bio-information gathered 
from multi-sensor array 50). 



Application/Control Number: 10/541,278 Page 13 

Art Unit: 2464 

For claim 11, Fernandez et al. discloses an information transmission information 
acquiring means wherein the at least one individual includes a listener who is present at the given 
location when the audio information is acquired and/or a viewer present at the given location 
place when the video information is acquired (see column 7 lines 1-9, which recite real-time 
conference that transmits information of the listener including bio-information gathered from 
multi-sensor array 50). 

For claim 15, Fernandez et al. discloses an information transmission information 
acquiring means wherein the bio-information is selected from the group consisting of body 
motion, myoelectricity, body surface temperature, skin sweating, skin resistance, pulse, breath, 
micro-vibration, cardioelectricity, heartbeat, and blood pressure (see column 4 lines 20-29, which 
recite detecting temperature, pressure, physiological vital information such as pulse and blood 
pressure using the multi-sensor module 50). 

For claim 16, Fernandez et al. discloses an information transmission information 
acquiring means wherein the bio-information detecting means extracts the detected bio- 
information from the one or more of audio information and video information (see column 6 
lines 22-25, which recite extracting facial imaging biometric information from video information 
gathered at client 20). 

For claim 17, Fernandez et al. disclose an information recording method, comprising: 

acquiring one or more of audio information and video information of a performance at a 
given location (see figure 2 and column 2 lines 23-34, which recite a microphone 44 and camera 
42 for acquiring audio and visual information); 
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detecting, concurrent with the acquiring of the one or more of audio information and 
video information, bio-information of at least one individual participator present at the given 
location (see figure 2,figure 4 and column 2 lines 23-34, which recite a biometric sensor array 
module 50 that detects bio-information); 

performing statistical processing of the bio-information (see column 3 lines 48-67 and 
column 4 lines 1-19, which recite statistical processing of the bio-information by collecting and 
interpreting the mix-mode signals of the bio sensors 50 into findings, flags, warnings, or other 
indications provided as feedback); 

packetizing the statistically processed bio-information into respective pluralities of 
second data packets corresponding to the respective pluralities of the first data packets (see 
column 4 lines 8-19, which recite generating digital packets comprising the collected and 
interpreted bio-information signals collected by biometric sensor array module 50); 

recording the multiplexed stream onto a predetermined recording medium (see column 2 
lines 4-14, which recite storing the recorded data for transmission). 

Fernandez et al. disclose all the subject of the claimed invention with the exception 
wherein the statistical processing of the bio-information is specifically performed over 
predetermined time intervals corresponding to respective pluralities of the first data packets of 
the one or more of audio information and video information, wherein the packetizing of 
statistically processed bio-information is specifically packetized into respective pluralities of 
second data packets corresponding to the respective pluralities of the first data packets. 

However, Kumar et al. disclose a system and method for real-time retrieval, assessment, 
analysis, and transmission of physiological data such as EKG, EMG, EEG, blood pressure, body 
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temperature, and pulse measurements (see column 5 lines 47-61 and column 6 lines 50-56). The 
real-time streaming of such bio-information is augmented by transmission of audio and video 
streams that are synchronized with the respective bio-information (see column 7 lines 4-14). 
Thus, it would have been obvious to the person of ordinary skill in the art at the time of the 
invention to use the system and method for communicating bio-information and respectively 
synchronized audio and video information as taught by Kumar et al. with the method for 
transmitting and receiving bio-information with audio and video information as taught by 
Fernandez et al. The packetized bio-information must be multiplexed and transmitted such that 
it is temporally adjacent to the audio and video packets to ensure that the bio-information is 
synchronized with the audio and video information to form a real-time combined data stream. If 
the bio-information and audio/video information of a corresponding time period are not 
multiplexed together, then one of the data information would arrive later. As a result, the final 
stream containing video, audio, and bio-information would be delayed until the late data arrives. 
In this case, the delay would prevent the real-time streaming of the combined data streams as 
disclosed by Kumar et al. Thus, the packetized bio-information must be multiplexed together 
with the audio and video packets to ensure that the bio-information is synchronized with the 
audio and video information to form a real-time combined data stream as disclosed by Kumar et 
al. The motivation for using the system and method for transmitting bio-information and 
respectively synchronized audio and video information as taught by Kumar et al. with the 
method for transmitting bio-information with audio and video information as taught by 
Fernandez et al. is to enhance the communication experienced by users using transmission of 
audio/video information with respectively synchronized bio-information. 
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Fernandez et al. and Kumar et al. disclose all the subject of the claimed invention with 
the exception of specifically partitioning the acquired one or more of audio information and 
video information into lengths of partitioned data each of which corresponds to a predefined time 
interval, compressing and packetizing the lengths of partitioned data into a stream of first data 
packets having empty regions resulting from the compression of lengths of the partitioned data, 
and multiplexing the stream of first data packets with the second data packets by inserting a 
respective plurality of second data packets, which corresponds to one of the predetermined time 
intervals, into one of the empty regions of the stream that is adjacent to a respective plurality of 
first data packets which corresponds to a same one of the predetermined time intervals. 

However, Stanger et al. from the same or similar fields of endeavor disclose a system and 
method for adding auxiliary data to empty space that results from compressing a data stream for 
transmission (see column 2 lines 7-37). Specifically, the system and method partitions video and 
audio data into data packets (see column 5 lines 10-27 and figures 3a-3b), compresses the 
packets to provide empty regions (see column 5 lines 31-65 and figure 4), and inserts auxiliary 
data in the empty spaces (see column 7 lines 41-65 and figure 5). Thus, it would have been 
obvious to the person of ordinary skill in the art at the time of the invention to use the system and 
method for adding auxiliary data to empty space that is the result of compressing a data stream as 
taught by Stanger et al. with the method for transmitting and receiving synchronized bio- 
information with audio and video information as taught by Fernandez et al. and Kumar et al. 

As noted above in reference to Kumar et al., the bio-information is synchronized with 
respective audio and video information to form a real-time data stream . Thus, bio-information 
must be multiplexed with audio/video information that all correspond to the same time period. If 
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the respective bio-information is not multiplexed with audio/video information that all 
correspond to the same time period, then one of the data information would arrive later. As a 
result, the final stream containing video, audio, and bio-information would be delayed until the 
late data arrives. In this case, the delay would prevent the real-time streaming of the combined 
data streams as disclosed by Kumar et al. Thus, the packetized bio-information of a time period 
must be multiplexed in empty regions that correspond to audio and video packets of the same 
time period to ensure that the bio-information is synchronized with the audio and video 
information to form a real-time combined data stream as disclosed by Kumar et al. The system 
and method for adding auxiliary data to empty space in a data stream can be implemented by 
inserting the synchronized bio-information as taught by Fernandez et al. and Kumar et al. to the 
empty space that results from compressing the audio and video data using the encoders and 
multiplexers as taught by Stanger et al. The motivation for using the system and method for 
adding auxiliary data to empty space that results from compressing a data stream as taught by 
Stanger et al. with the method for transmitting and receiving synchronized bio-information with 
audio and video information as taught by Fernandez et al. and Kumar et al. is to improve the 
efficiency of the system by creating more payload capacity in the existing transmission channels. 

For claim 18, Fernandez et al. discloses an information recording method wherein the at 
least one individual includes a speaker, a player, an actor, or conductor who serves as a source of 
the audio information, and/or a performer or a photographed person who is included within the 
video information (see column 7 lines 1-9, which recite real-time conference that transmits 
information of the speaker including bio-information gathered from multi-sensor array 50). 
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For claim 19, Fernandez et al. discloses an information recording method wherein the at 
least one individual includes a listener who is present at the given location when the audio 
information is acquired and/or a viewer present at the given location when the 
video information is acquired (see column 7 lines 1-9, which recite real-time conference that 
transmits information of the listener including bio-information gathered from multi-sensor array 
50). 

For claim 23, Fernandez et al. discloses an information recording method wherein the 
bio-information is selected from the group consisting of body motion, myoelectricity, body 
surface temperature, skin sweating, skin resistance, pulse, breath, micro-vibration, 
cardioelectricity, heartbeat, and blood pressure (see column 4 lines 20-29, which recite detecting 
temperature, pressure, physiological vital information such as pulse and blood pressure using 
the multi-sensor module 50). 

For claim 24, Fernandez et al. discloses an information recording method wherein the 
detected the recording medium is at least one of optical disc, magnetic tape, hard disc and 
semiconductor memory (see column 3 lines 1-17, which recite a memory implement on a 
semiconductor for storing sensed signals). 

For claim 25, Fernandez et al. discloses an information recording method wherein the 
bio-information is extracted from the one or more of audio information and video information 
(see column 6 lines 22-25, which recite extracting facial imaging biometric information from 
video information gathered at client 20). 

For claim 26, Fernandez et al. disclose an information recording apparatus, comprising: 
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information acquiring means for acquiring one or more of audio information and video 
information of a performance at a given location (see figure 2 and column 2 lines 23-34, which 
recite a microphone 44 and camera 42 for acquiring audio and visual information); 

bio-information detecting means for detecting, concurrent with the acquiring of the one 
or more of audio information and video information, bio-information of at least one individual 
present at the given location (see figure 2,figure 4 and column 2 lines 23-34, which recite a 
biometric sensor array module 50 that detects bio-information); and 

first packetizing means for packetizing the acquired one or more of audio information 
and video information into a stream of first data packets (see column 1 lines 62-65 and figure 2, 
which recite using TCP/IP packets to transmit audio and video information gathered by 
microphone 44 and camera 42 from client 20 to network 10); 

bio-information analysis means for performing statistical processing of the bio- 
information (see column 3 lines 48-67 and column 4 lines 1-19, which recite statistical 
processing of the bio-information by collecting and interpreting the mix-mode signals of the bio 
sensors 50 into findings, flags, warnings, or other indications provided as feedback); 

second packetizing means for packetizing the statistically processed bio-information into 
respective pluralities of second data packets (see column 4 lines 8-19, which recite generating 
digital packets comprising the collected and interpreted bio-information signals collected by 
biometric sensor array module 50); 

recording means for recording the multiplexed stream onto a predetermined recording 
medium (see column 2 lines 4-14, which recite storing the recorded data for transmission). 
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Fernandez et al. disclose all the subject of the claimed invention with the exception 
wherein the statistical processing of the bio-information is specifically performed over 
predetermined time intervals corresponding to respective pluralities of the first data packets of 
the one or more of audio information and video information, wherein the packetizing of 
statistically processed bio-information is specifically packetized into respective pluralities of 
second data packets corresponding to the respective pluralities of the first data packets. 

However, Kumar et al. disclose a system and method for real-time retrieval, assessment, 
analysis, and transmission of physiological data such as EKG, EMG, EEG, blood pressure, body 
temperature, and pulse measurements (see column 5 lines 47-61 and column 6 lines 50-56). The 
real-time streaming of such bio-information is augmented by transmission of audio and video 
streams that are synchronized with the respective bio-information (see column 7 lines 4-14). 
Thus, it would have been obvious to the person of ordinary skill in the art at the time of the 
invention to use the system and method for communicating bio-information and respectively 
synchronized audio and video information as taught by Kumar et al. with the method for 
transmitting and receiving bio-information with audio and video information as taught by 
Fernandez et al. The packetized bio-information must be multiplexed and transmitted such that 
it is temporally adjacent to the audio and video packets to ensure that the bio-information is 
synchronized with the audio and video information to form a real-time combined data stream. If 
the bio-information and audio/video information of a corresponding time period are not 
multiplexed together, then one of the data information would arrive later. As a result, the final 
stream containing video, audio, and bio-information would be delayed until the late data arrives. 
In this case, the delay would prevent the real-time streaming of the combined data streams as 
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disclosed by Kumar et al. Thus, the packetized bio-information must be multiplexed together 
with the audio and video packets to ensure that the bio-information is synchronized with the 
audio and video information to form a real-time combined data stream as disclosed by Kumar et 
al. The motivation for using the system and method for transmitting bio-information and 
respectively synchronized audio and video information as taught by Kumar et al. with the 
method for transmitting bio-information with audio and video information as taught by 
Fernandez et al. is to enhance the communication experienced by users using transmission of 
audio/video information with respectively synchronized bio-information. 

Fernandez et al. and Kumar et al. disclose all the subject of the claimed invention with 
the exception of specifically partitioning the acquired one or more of audio information and 
video information into lengths of partitioned data each of which corresponds to a predefined time 
interval, compressing and packetizing the lengths of partitioned data into a stream of first data 
packets having empty regions resulting from the compression of lengths of the partitioned data, 
and multiplexing the stream of first data packets with the second data packets by inserting a 
respective plurality of second data packets, which corresponds to one of the predetermined time 
intervals, into one of the empty regions of the stream that is adjacent to a respective plurality of 
first data packets which corresponds to a same one of the predetermined time intervals. 

However, Stanger et al. from the same or similar fields of endeavor disclose a system and 
method for adding auxiliary data to empty space that results from compressing a data stream for 
transmission (see column 2 lines 7-37). Specifically, the system and method partitions video and 
audio data into data packets (see column 5 lines 10-27 and figures 3a-3b), compresses the 
packets to provide empty regions (see column 5 lines 31-65 and figure 4), and inserts auxiliary 
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data in the empty spaces (see column 7 lines 41-65 and figure 5). Thus, it would have been 
obvious to the person of ordinary skill in the art at the time of the invention to use the system and 
method for adding auxiliary data to empty space that is the result of compressing a data stream as 
taught by Stanger et al. with the method for transmitting and receiving synchronized bio- 
information with audio and video information as taught by Fernandez et al. and Kumar et al. 

As noted above in reference to Kumar et al., the bio-information is synchronized with 
respective audio and video information to form a real-time data stream . Thus, bio-information 
must be multiplexed with audio/video information that all correspond to the same time period. If 
the respective bio-information is not multiplexed with audio/video information that all 
correspond to the same time period, then one of the data information would arrive later. As a 
result, the final stream containing video, audio, and bio-information would be delayed until the 
late data arrives. In this case, the delay would prevent the real-time streaming of the combined 
data streams as disclosed by Kumar et al. Thus, the packetized bio-information of a time period 
must be multiplexed in empty regions that correspond to audio and video packets of the same 
time period to ensure that the bio-information is synchronized with the audio and video 
information to form a real-time combined data stream as disclosed by Kumar et al. The system 
and method for adding auxiliary data to empty space in a data stream can be implemented by 
inserting the synchronized bio-information as taught by Fernandez et al. and Kumar et al. to the 
empty space that results from compressing the audio and video data using the encoders and 
multiplexers as taught by Stanger et al. The motivation for using the system and method for 
adding auxiliary data to empty space that results from compressing a data stream as taught by 
Stanger et al. with the method for transmitting and receiving synchronized bio-information with 
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audio and video information as taught by Fernandez et al. and Kumar et al. is to improve the 
efficiency of the system by creating more payload capacity in the existing transmission channels. 

For claim 27, Fernandez et al. discloses an information recording method wherein the at 
least one individual includes a speaker, a player, an actor, or a conductor who serves as a source 
of the audio information, and/or a performer or a photographed person who is included within 
the video information (see column 7 lines 1-9, which recite real-time conference that transmits 
information of the speaker including bio-information gathered from multi-sensor array 50). 

For claim 28, Fernandez et al. discloses an information recording method wherein the at 
least one individual includes a listener who is present at the given location when the audio 
information is acquired and/or a viewer present at the given location when the video information 
is acquired (see column 7 lines 1-9, which recite real-time conference that transmits information 
of the listener including bio-information gathered from multi-sensor array 50). 

For claim 32, Fernandez et al. discloses an information recording method wherein the 
bio-information is at least one selected from the group consisting of body motion, myoelectricity, 
body surface temperature, skin sweating, skin resistance, pulse, breath, micro-vibration, 
cardioelectricity, heartbeat, and blood pressure (see column 4 lines 20-29, which recite detecting 
temperature, pressure, physiological vital information such as pulse and blood pressure using 
the multi-sensor module 50). 

For claim 33, Fernandez et al. discloses an information recording method wherein the 
recording medium is selected from the group consisting of optical disc, magnetic tape, hard disc, 
and semiconductor memory (see column 3 lines 1-17, which recite a memory implement on a 
semiconductor for storing sensed signals). 
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For claim 34, Fernandez et al. discloses an information recording method wherein the 
bio-information detecting means extracts the detected bio-information from the one or more of 
audio information and video information (see column 6 lines 22-25, which recite extracting 
facial imaging biometric information from video information gathered at client 20). 

For claim 35, Fernandez et al. disclose an information reproducing method, comprising: 

decomposing a multiplexed data stream into respective pluralities of first data packets of 
one or more of audio information and video information and into corresponding respective 
pluralities of second data packets of statistically processed bio-information (see figure 2 and 
column 2 lines 23-34, which recites client 20 including network interface 24 used to receive data 
streams including audio, video, and bio-information that are decomposed and respectively 
distributed to audio device 46, display 32, and mechanical device 48), 

the multiplexed data stream having each respective plurality of second data packets of the 
statistically processed bio-information (see column 4 lines 8-19 and column 7 lines 1-9, which 
recite generating digital packets comprising the collected and interpreted bio-information 
signals collected by biometric sensor array module 50 for real-time exchange of live audio/video 
and the collected bio-information between clients), 

the one or more of audio information and video information being acquired at a 
performance at a given location (see figure 2 and column 2 lines 23-34, which recite a 
microphone 44 and camera 42 for acquiring audio and visual information), 

the bio-information being of at least one individual present at the given location and 
being detected concurrent with the acquiring of the one or more of audio information and video 
information (see figure 2, figure 4, column 2 lines 23-34, and column 7 lines 1-9, which recite a 
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biometric sensor array module 50 that detects bio-information used for real-time conference that 
concurrently transmits audio/video information as well as bio-information gathered from multi- 
sensor array 50); 

the statistically processed bio-information being generated by performing statistical 
processing of the detected bio-information over predetermined time intervals corresponding to 
respective pluralities of the first data packets (see column 3 lines 48-67 and column 4 lines 1-19, 
which recite statistical processing of the bio-information by collecting and interpreting the mix- 
mode signals of the bio sensors 50 into findings, flags, warnings, or other indications provided 
as feedback); 

reproducing the one or more of audio information and video information for delivery to a 
user (see figure 2 and column 2 lines 23-34, which recites client 20 including network interface 
24 for receiving a data stream that are decomposed and distributed to the display 32 and audio 
device 46); and 

providing, to the user, sense stimulation based on the bio-information concurrent with the 
delivery of the one or more of the audio information and video information (see figure 2 and 
column 2 lines 23-34, which recites client 20 including network interface 24 for receiving a data 
stream that are decomposed and distributed to mechanical device 48 for providing sense 
stimulation to a user). 

Fernandez et al. disclose all the subject of the claimed invention with the exception 
wherein the statistical processing of the bio-information is specifically performed over 
predetermined time intervals corresponding to respective pluralities of the first data packets of 
the one or more of audio information and video information, wherein the packetizing of 
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statistically processed bio-information is specifically packetized into respective pluralities of 
second data packets corresponding to the respective pluralities of the first data packets. 

However, Kumar et al. disclose a system and method for real-time retrieval, assessment, 
analysis, and transmission of physiological data such as EKG, EMG, EEG, blood pressure, body 
temperature, and pulse measurements (see column 5 lines 47-61 and column 6 lines 50-56). The 
real-time streaming of such bio-information is augmented by transmission of audio and video 
streams that are synchronized with the respective bio-information (see column 7 lines 4-14). 
Thus, it would have been obvious to the person of ordinary skill in the art at the time of the 
invention to use the system and method for communicating bio-information and respectively 
synchronized audio and video information as taught by Kumar et al. with the method for 
transmitting and receiving bio-information with audio and video information as taught by 
Fernandez et al. The packetized bio-information must be multiplexed and transmitted such that 
it is temporally adjacent to the audio and video packets to ensure that the bio-information is 
synchronized with the audio and video information to form a real-time combined data stream. If 
the bio-information and audio/video information of a corresponding time period are not 
multiplexed together, then one of the data information would arrive later. As a result, the final 
stream containing video, audio, and bio-information would be delayed until the late data arrives. 
In this case, the delay would prevent the real-time streaming of the combined data streams as 
disclosed by Kumar et al. Thus, the packetized bio-information must be multiplexed together 
with the audio and video packets to ensure that the bio-information is synchronized with the 
audio and video information to form a real-time combined data stream as disclosed by Kumar et 
al. The motivation for using the system and method for transmitting bio-information and 
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respectively synchronized audio and video information as taught by Kumar et al. with the 
method for transmitting bio-information with audio and video information as taught by 
Fernandez et al. is to enhance the communication experienced by users using transmission of 
audio/video information with respectively synchronized bio-information. 

Fernandez et al. and Kumar et al. disclose all the subject of the claimed invention with 
the exception of specifically partitioning the acquired one or more of audio information and 
video information into lengths of partitioned data each of which corresponds to a predefined time 
interval, compressing and packetizing the lengths of partitioned data into a stream of first data 
packets having empty regions resulting from the compression of lengths of the partitioned data, 
and multiplexing the stream of first data packets with the second data packets by inserting a 
respective plurality of second data packets, which corresponds to one of the predetermined time 
intervals, into one of the empty regions of the stream that is adjacent to a respective plurality of 
first data packets which corresponds to a same one of the predetermined time intervals. 

However, Stanger et al. from the same or similar fields of endeavor disclose a system and 
method for adding auxiliary data to empty space that results from compressing a data stream for 
transmission (see column 2 lines 7-37). Specifically, the system and method partitions video and 
audio data into data packets (see column 5 lines 10-27 and figures 3a-3b), compresses the 
packets to provide empty regions (see column 5 lines 31-65 and figure 4), and inserts auxiliary 
data in the empty spaces (see column 7 lines 41-65 and figure 5). Thus, it would have been 
obvious to the person of ordinary skill in the art at the time of the invention to use the system and 
method for adding auxiliary data to empty space that is the result of compressing a data stream as 
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taught by Stanger et al. with the method for transmitting and receiving synchronized bio- 
information with audio and video information as taught by Fernandez et al. and Kumar et al. 

As noted above in reference to Kumar et al., the bio-information is synchronized with 
respective audio and video information to form a real-time data stream . Thus, bio-information 
must be multiplexed with audio/video information that all correspond to the same time period. If 
the respective bio-information is not multiplexed with audio/video information that all 
correspond to the same time period, then one of the data information would arrive later. As a 
result, the final stream containing video, audio, and bio-information would be delayed until the 
late data arrives. In this case, the delay would prevent the real-time streaming of the combined 
data streams as disclosed by Kumar et al. Thus, the packetized bio-information of a time period 
must be multiplexed in empty regions that correspond to audio and video packets of the same 
time period to ensure that the bio-information is synchronized with the audio and video 
information to form a real-time combined data stream as disclosed by Kumar et al. The system 
and method for adding auxiliary data to empty space in a data stream can be implemented by 
inserting the synchronized bio-information as taught by Fernandez et al. and Kumar et al. to the 
empty space that results from compressing the audio and video data using the encoders and 
multiplexers as taught by Stanger et al. The motivation for using the system and method for 
adding auxiliary data to empty space that results from compressing a data stream as taught by 
Stanger et al. with the method for transmitting and receiving synchronized bio-information with 
audio and video information as taught by Fernandez et al. and Kumar et al. is to improve the 
efficiency of the system by creating more payload capacity in the existing transmission channels. 
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For claim 36, Fernandez et al. discloses an information reproducing method wherein the 
multiplexed data stream is received through a transmission method (see figure 2, which recites a 
client 20 that uses network interface 24 for receiving the sensed data stream). 

For claim 37, Fernandez et al. discloses an information reproducing method wherein the 
multiplexed data stream is read out from a recording medium (see column 3 lines 1-17, which 
recite a memory implement on a semiconductor for reading out sensed signals). 

For claim 38, Fernandez et al. discloses an information reproducing method wherein the 
at least one individual includes a speaker, a player, an actor or conductor who serves as a source 
of the audio information, and/or a performer or a photographed person who is included within 
the video information (see column 7 lines 1-9, which recite real-time conference that transmits 
information of the speaker including bio-information gathered from multi-sensor array 50). 

For claim 39, Fernandez et al. discloses an information reproducing method wherein the 
at least one individual includes a listener present at the given location when the audio 
information is acquired and/or a viewer present at the given location when the video information 
is acquired (see column 7 lines 1-9, which recite real-time conference that transmits information 
of the listener including bio-information gathered from multi-sensor array 50). 

For claim 40, Fernandez et al. disclose an information reproducing method, comprising: 

decomposing a multiplexed data stream into respective pluralities of first data packets of 
one or more of audio information and video information and into corresponding respective 
pluralities of second data packets of statistically processed bio-information (see figure 2 and 
column 2 lines 23-34, which recites client 20 including network interface 24 used to receive data 
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streams including audio, video, and bio-information that are decomposed and respectively 
distributed to audio device 46, display 32, and mechanical device 48), 

the multiplexed data stream having each respective plurality of second data packets of the 
statistically processed bio-information (see column 4 lines 8-19 and column 7 lines 1-9, which 
recite generating digital packets comprising the collected and interpreted bio-information 
signals collected by biometric sensor array module 50 for real-time exchange of live audio/video 
and the collected bio-information between clients), 

the one or more of audio information and video information being acquired at a 
performance at a given location (see figure 2 and column 2 lines 23-34, which recite a 
microphone 44 and camera 42 for acquiring audio and visual information), 

the bio-information being of at least one individual present at the given location and 
being detected concurrent with the acquiring of the one or more of audio information and video 
information (see figure 2, figure 4, column 2 lines 23-34, and column 7 lines 1-9, which recite a 
biometric sensor array module 50 that detects bio-information used for real-time conference that 
concurrently transmits audio/video information as well as bio-information gathered from multi- 
sensor array 50), 

the statistically processed bio-information being generated by performing statistical 
processing of the detected bio-information over predetermined time intervals corresponding to 
respective pluralities of the first data packets (see column 3 lines 48-67 and column 4 lines 1-19, 
which recite statistical processing of the bio-information by collecting and interpreting the mix- 
mode signals of the bio sensors 50 into findings, flags, warnings, or other indications provided 
as feedback); and 



Application/Control Number: 10/541,278 Page 31 

Art Unit: 2464 

controlling, based on the bio-information, reproduction of the one or more of audio 
information and video information (see column 6 lines 22-25, which controlling access to video 
information based upon facial imaging biometric information). 

Fernandez et al. disclose all the subject of the claimed invention with the exception 
wherein the statistical processing of the bio-information is specifically performed over 
predetermined time intervals corresponding to respective pluralities of the first data packets of 
the one or more of audio information and video information, wherein the packetizing of 
statistically processed bio-information is specifically packetized into respective pluralities of 
second data packets corresponding to the respective pluralities of the first data packets. 

However, Kumar et al. disclose a system and method for real-time retrieval, assessment, 
analysis, and transmission of physiological data such as EKG, EMG, EEG, blood pressure, body 
temperature, and pulse measurements (see column 5 lines 47-61 and column 6 lines 50-56). The 
real-time streaming of such bio-information is augmented by transmission of audio and video 
streams that are synchronized with the respective bio-information (see column 7 lines 4-14). 
Thus, it would have been obvious to the person of ordinary skill in the art at the time of the 
invention to use the system and method for communicating bio-information and respectively 
synchronized audio and video information as taught by Kumar et al. with the method for 
transmitting and receiving bio-information with audio and video information as taught by 
Fernandez et al. The packetized bio-information must be multiplexed and transmitted such that 
it is temporally adjacent to the audio and video packets to ensure that the bio-information is 
synchronized with the audio and video information to form a real-time combined data stream. If 
the bio-information and audio/video information of a corresponding time period are not 
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multiplexed together, then one of the data information would arrive later. As a result, the final 
stream containing video, audio, and bio-information would be delayed until the late data arrives. 
In this case, the delay would prevent the real-time streaming of the combined data streams as 
disclosed by Kumar et al. Thus, the packetized bio-information must be multiplexed together 
with the audio and video packets to ensure that the bio-information is synchronized with the 
audio and video information to form a real-time combined data stream as disclosed by Kumar et 
al. The motivation for using the system and method for transmitting bio-information and 
respectively synchronized audio and video information as taught by Kumar et al. with the 
method for transmitting bio-information with audio and video information as taught by 
Fernandez et al. is to enhance the communication experienced by users using transmission of 
audio/video information with respectively synchronized bio-information. 

Fernandez et al. and Kumar et al. disclose all the subject of the claimed invention with 
the exception of specifically partitioning the acquired one or more of audio information and 
video information into lengths of partitioned data each of which corresponds to a predefined time 
interval, compressing and packetizing the lengths of partitioned data into a stream of first data 
packets having empty regions resulting from the compression of lengths of the partitioned data, 
and multiplexing the stream of first data packets with the second data packets by inserting a 
respective plurality of second data packets, which corresponds to one of the predetermined time 
intervals, into one of the empty regions of the stream that is adjacent to a respective plurality of 
first data packets which corresponds to a same one of the predetermined time intervals. 

However, Stanger et al. from the same or similar fields of endeavor disclose a system and 
method for adding auxiliary data to empty space that results from compressing a data stream for 
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transmission (see column 2 lines 7-37). Specifically, the system and method partitions video and 
audio data into data packets (see column 5 lines 10-27 and figures 3a-3b), compresses the 
packets to provide empty regions (see column 5 lines 31-65 and figure 4), and inserts auxiliary 
data in the empty spaces (see column 7 lines 41-65 and figure 5). Thus, it would have been 
obvious to the person of ordinary skill in the art at the time of the invention to use the system and 
method for adding auxiliary data to empty space that is the result of compressing a data stream as 
taught by Stanger et al. with the method for transmitting and receiving synchronized bio- 
information with audio and video information as taught by Fernandez et al. and Kumar et al. 

As noted above in reference to Kumar et al., the bio-information is synchronized with 
respective audio and video information to form a real-time data stream . Thus, bio-information 
must be multiplexed with audio/video information that all correspond to the same time period. If 
the respective bio-information is not multiplexed with audio/video information that all 
correspond to the same time period, then one of the data information would arrive later. As a 
result, the final stream containing video, audio, and bio-information would be delayed until the 
late data arrives. In this case, the delay would prevent the real-time streaming of the combined 
data streams as disclosed by Kumar et al. Thus, the packetized bio-information of a time period 
must be multiplexed in empty regions that correspond to audio and video packets of the same 
time period to ensure that the bio-information is synchronized with the audio and video 
information to form a real-time combined data stream as disclosed by Kumar et al. The system 
and method for adding auxiliary data to empty space in a data stream can be implemented by 
inserting the synchronized bio-information as taught by Fernandez et al. and Kumar et al. to the 
empty space that results from compressing the audio and video data using the encoders and 
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multiplexers as taught by Stanger et al. The motivation for using the system and method for 
adding auxiliary data to empty space that results from compressing a data stream as taught by 
Stanger et al. with the method for transmitting and receiving synchronized bio-information with 
audio and video information as taught by Fernandez et al. and Kumar et al. is to improve the 
efficiency of the system by creating more payload capacity in the existing transmission channels. 

For claim 41, Fernandez et al. discloses an information reproducing method wherein the 
multiplexed data stream is received through a transmission medium (see figure 2, which recites a 
client 20 that uses network interface 24 for receiving the sensed data stream). 

For claim 42, Fernandez et al. discloses an information reproducing method 
wherein the multiplexed data stream is read out from a record medium (see column 3 
lines 1-17, which recite a memory implement on a semiconductor for reading out sensed 
signals). 

For claim 43, Fernandez et al. discloses an information reproducing method wherein the 
at least one individual includes a speaker, a player, an actor, or conductor who serves as a source 
of the audio information, and/or a performer or a photographed person who is included within 
the video information (see column 7 lines 1-9, which recite real-time conference that transmits 
information of the speaker including bio-information gathered from multi-sensor array 50). 

For claim 44, Fernandez et al. discloses an information reproducing method wherein the 
at least one individual includes a listener present at the given location place when the audio 
information is acquired and/or a viewer present at the given location place when the video 
information is acquired (see column 7 lines 1-9, which recite real-time conference that transmits 
information of the listener including bio-information gathered from multi-sensor array 50). 
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For claim 45, Fernandez et al. disclose an information reproducing apparatus, 
comprising: 

means for decomposing a multiplexed data stream into respective pluralities of first data 
packets of one or more of audio information and video information and into corresponding 
respective pluralities of second data packets of statistically processed bio-information (see figure 
2 and column 2 lines 23-34, which recites client 20 including network interface 24 used to 
receive data streams including audio, video, and bio-information that are decomposed and 
respectively distributed to audio device 46, display 32, and mechanical device 48), 

the multiplexed data stream having each respective plurality of second data packets of the 
statistically processed bio-information (see column 4 lines 8-19 and column 7 lines 1-9, which 
recite generating digital packets comprising the collected and interpreted bio-information 
signals collected by biometric sensor array module 50 for real-time exchange of live audio/video 
and the collected bio-information between clients), 

the one or more of audio information and video information being acquired at a 
performance at a given location (see figure 2 and column 2 lines 23-34, which recite a 
microphone 44 and camera 42 for acquiring audio and visual information), 

the bio-information being of at least one individual present at the given location and 
being detected concurrent with the acquiring of the one or more of audio information and 
video information (see figure 2, figure 4, column 2 lines 23-34, and column 7 lines 1-9, which 
recite a biometric sensor array module 50 that detects bio-information used for real-time 
conference that concurrently transmits audio/video information as well as bio-information 
gathered from multi-sensor array 50); 



Application/Control Number: 10/541,278 Page 36 

Art Unit: 2464 

the statistically processed bio-information being generated by performing statistical 
processing of the detected bio-information over predetermined time intervals corresponding to 
respective pluralities of the first data packets (see column 3 lines 48-67 and column 4 lines 1-19, 
which recite statistical processing of the bio-information by collecting and interpreting the mix- 
mode signals of the bio sensors 50 into findings, flags, warnings, or other indications provided 
as feedback); 

means for reproducing the one or more of audio information and video information for 
delivery to a user (see figure 2 and column 2 lines 23-34, which recites client 20 including 
network interface 24 for receiving a data stream that are decomposed and distributed to the 
display 32 and audio device 46); and 

means for providing, to the user, sense stimulation based on the bio-information 
concurrent with the delivery of the one or more of the audio information and video information 
(see figure 2 and column 2 lines 23-34, which recites client 20 including network interface 24 for 
receiving a data stream that are decomposed and distributed to mechanical device 48 for 
providing sense stimulation to a user). 

Fernandez et al. disclose all the subject of the claimed invention with the exception 
wherein the statistical processing of the bio-information is specifically performed over 
predetermined time intervals corresponding to respective pluralities of the first data packets of 
the one or more of audio information and video information, wherein the packetizing of 
statistically processed bio-information is specifically packetized into respective pluralities of 
second data packets corresponding to the respective pluralities of the first data packets. 
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However, Kumar et al. disclose a system and method for real-time retrieval, assessment, 
analysis, and transmission of physiological data such as EKG, EMG, EEG, blood pressure, body 
temperature, and pulse measurements (see column 5 lines 47-61 and column 6 lines 50-56). The 
real-time streaming of such bio-information is augmented by transmission of audio and video 
streams that are synchronized with the respective bio-information (see column 7 lines 4-14). 
Thus, it would have been obvious to the person of ordinary skill in the art at the time of the 
invention to use the system and method for communicating bio-information and respectively 
synchronized audio and video information as taught by Kumar et al. with the method for 
transmitting and receiving bio-information with audio and video information as taught by 
Fernandez et al. The packetized bio-information must be multiplexed and transmitted such that 
it is temporally adjacent to the audio and video packets to ensure that the bio-information is 
synchronized with the audio and video information to form a real-time combined data stream. If 
the bio-information and audio/video information of a corresponding time period are not 
multiplexed together, then one of the data information would arrive later. As a result, the final 
stream containing video, audio, and bio-information would be delayed until the late data arrives. 
In this case, the delay would prevent the real-time streaming of the combined data streams as 
disclosed by Kumar et al. Thus, the packetized bio-information must be multiplexed together 
with the audio and video packets to ensure that the bio-information is synchronized with the 
audio and video information to form a real-time combined data stream as disclosed by Kumar et 
al. The motivation for using the system and method for transmitting bio-information and 
respectively synchronized audio and video information as taught by Kumar et al. with the 
method for transmitting bio-information with audio and video information as taught by 
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Fernandez et al. is to enhance the communication experienced by users using transmission of 
audio/video information with respectively synchronized bio-information. 

Fernandez et al. and Kumar et al. disclose all the subject of the claimed invention with 
the exception of specifically partitioning the acquired one or more of audio information and 
video information into lengths of partitioned data each of which corresponds to a predefined time 
interval, compressing and packetizing the lengths of partitioned data into a stream of first data 
packets having empty regions resulting from the compression of lengths of the partitioned data, 
and multiplexing the stream of first data packets with the second data packets by inserting a 
respective plurality of second data packets, which corresponds to one of the predetermined time 
intervals, into one of the empty regions of the stream that is adjacent to a respective plurality of 
first data packets which corresponds to a same one of the predetermined time intervals. 

However, Stanger et al. from the same or similar fields of endeavor disclose a system and 
method for adding auxiliary data to empty space that results from compressing a data stream for 
transmission (see column 2 lines 7-37). Specifically, the system and method partitions video and 
audio data into data packets (see column 5 lines 10-27 and figures 3a-3b), compresses the 
packets to provide empty regions (see column 5 lines 31-65 and figure 4), and inserts auxiliary 
data in the empty spaces (see column 7 lines 41-65 and figure 5). Thus, it would have been 
obvious to the person of ordinary skill in the art at the time of the invention to use the system and 
method for adding auxiliary data to empty space that is the result of compressing a data stream as 
taught by Stanger et al. with the method for transmitting and receiving synchronized bio- 
information with audio and video information as taught by Fernandez et al. and Kumar et al. 
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As noted above in reference to Kumar et al., the bio-information is synchronized with 
respective audio and video information to form a real-time data stream . Thus, bio-information 
must be multiplexed with audio/video information that all correspond to the same time period. If 
the respective bio-information is not multiplexed with audio/video information that all 
correspond to the same time period, then one of the data information would arrive later. As a 
result, the final stream containing video, audio, and bio-information would be delayed until the 
late data arrives. In this case, the delay would prevent the real-time streaming of the combined 
data streams as disclosed by Kumar et al. Thus, the packetized bio-information of a time period 
must be multiplexed in empty regions that correspond to audio and video packets of the same 
time period to ensure that the bio-information is synchronized with the audio and video 
information to form a real-time combined data stream as disclosed by Kumar et al. The system 
and method for adding auxiliary data to empty space in a data stream can be implemented by 
inserting the synchronized bio-information as taught by Fernandez et al. and Kumar et al. to the 
empty space that results from compressing the audio and video data using the encoders and 
multiplexers as taught by Stanger et al. The motivation for using the system and method for 
adding auxiliary data to empty space that results from compressing a data stream as taught by 
Stanger et al. with the method for transmitting and receiving synchronized bio-information with 
audio and video information as taught by Fernandez et al. and Kumar et al. is to improve the 
efficiency of the system by creating more payload capacity in the existing transmission channels. 

For claim 46, Fernandez et al. discloses an information reproducing apparatus further 
comprising: means for receiving the multiplexed data stream through a transmission medium 
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(see figure 2, which recites a client 20 that uses network interface 24 for receiving the sensed 
data stream). 

For claim 47, Fernandez et al. discloses an information reproducing apparatus further 
comprising: means for reading out the multiplexed data stream from a recording medium (see 
column 3 lines 1-17, which recite a memory implement on a semiconductor for reading out 
sensed signals). 

For claim 48, Fernandez et al. discloses an information reproducing apparatus wherein 
the at least one individual includes a speaker, a player, an actor, or a conductor who serves as a 
source of the audio information, and/or a performer or a photographed person who is included 
within the video information (see column 7 lines 1-9, which recite real-time conference that 
transmits information of the speaker including bio-information gathered from multi-sensor array 
50). 

For claim 49, Fernandez et al. discloses an information reproducing apparatus wherein 
the at least one individual includes a listener present at the given location when the audio 
information is acquired and/or a viewer present at the given location when video information is 
acquired (see column 7 lines 1-9, which recite real-time conference that transmits information of 
the listener including bio-information gathered from multi-sensor array 50). 

For claim 50, Fernandez et al. disclose an information reproducing apparatus, 
comprising: 

means for decomposing a multiplexed data stream into respective pluralities of first data 
packets of one or more of audio information and video information and into corresponding 
respective pluralities of second data packets of statistically processed bio-information (see figure 
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2 and column 2 lines 23-34, which recites client 20 including network interface 24 used to 
receive data streams including audio, video, and bio-information that are decomposed and 
respectively distributed to audio device 46, display 32, and mechanical device 48), 

the multiplexed data stream having each respective plurality of second data packets of the 
statistically processed bio-information (see column 4 lines 8-19 and column 7 lines 1-9, which 
recite generating digital packets comprising the collected and interpreted bio-information 
signals collected by biometric sensor array module 50 for real-time exchange of live audio/video 
and the collected bio-information between clients), 

the one or more of audio information and video information being acquired at a 
performance at a given location (see figure 2 and column 2 lines 23-34, which recite a 
microphone 44 and camera 42 for acquiring audio and visual information), 

the bio-information being of at least one individual present at the given location and 
being detected concurrent with the acquiring of the one or more of audio information and 
video information (see figure 2, figure 4, column 2 lines 23-34, and column 7 lines 1-9, which 
recite a biometric sensor array module 50 that detects bio-information used for real-time 
conference that concurrently transmits audio/video information as well as bio-information 
gathered from multi-sensor array 50), 

the statistically processed bio-information being generated by performing statistical 
processing of the detected bio-information over predetermined time intervals corresponding to 
respective pluralities of the first data packets (see column 3 lines 48-67 and column 4 lines 1-19, 
which recite statistical processing of the bio-information by collecting and interpreting the mix- 
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mode signals of the bio sensors 50 into findings, flags, warnings, or other indications provided 
as feedback); and 

means for controlling, based on the bio-information, reproduction of the one or more of 
audio information and video information (see column 6 lines 22-25, which controlling access to 
video information based upon facial imaging biometric information). 

Fernandez et al. disclose all the subject of the claimed invention with the exception 
wherein the statistical processing of the bio-information is specifically performed over 
predetermined time intervals corresponding to respective pluralities of the first data packets of 
the one or more of audio information and video information, wherein the packetizing of 
statistically processed bio-information is specifically packetized into respective pluralities of 
second data packets corresponding to the respective pluralities of the first data packets. 

However, Kumar et al. disclose a system and method for real-time retrieval, assessment, 
analysis, and transmission of physiological data such as EKG, EMG, EEG, blood pressure, body 
temperature, and pulse measurements (see column 5 lines 47-61 and column 6 lines 50-56). The 
real-time streaming of such bio-information is augmented by transmission of audio and video 
streams that are synchronized with the respective bio-information (see column 7 lines 4-14). 
Thus, it would have been obvious to the person of ordinary skill in the art at the time of the 
invention to use the system and method for communicating bio-information and respectively 
synchronized audio and video information as taught by Kumar et al. with the method for 
transmitting and receiving bio-information with audio and video information as taught by 
Fernandez et al. The packetized bio-information must be multiplexed and transmitted such that 
it is temporally adjacent to the audio and video packets to ensure that the bio-information is 
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synchronized with the audio and video information to form a real-time combined data stream. If 
the bio-information and audio/video information of a corresponding time period are not 
multiplexed together, then one of the data information would arrive later. As a result, the final 
stream containing video, audio, and bio-information would be delayed until the late data arrives. 
In this case, the delay would prevent the real-time streaming of the combined data streams as 
disclosed by Kumar et al. Thus, the packetized bio-information must be multiplexed together 
with the audio and video packets to ensure that the bio-information is synchronized with the 
audio and video information to form a real-time combined data stream as disclosed by Kumar et 
al. The motivation for using the system and method for transmitting bio-information and 
respectively synchronized audio and video information as taught by Kumar et al. with the 
method for transmitting bio-information with audio and video information as taught by 
Fernandez et al. is to enhance the communication experienced by users using transmission of 
audio/video information with respectively synchronized bio-information. 

Fernandez et al. and Kumar et al. disclose all the subject of the claimed invention with 
the exception of specifically partitioning the acquired one or more of audio information and 
video information into lengths of partitioned data each of which corresponds to a predefined time 
interval, compressing and packetizing the lengths of partitioned data into a stream of first data 
packets having empty regions resulting from the compression of lengths of the partitioned data, 
and multiplexing the stream of first data packets with the second data packets by inserting a 
respective plurality of second data packets, which corresponds to one of the predetermined time 
intervals, into one of the empty regions of the stream that is adjacent to a respective plurality of 
first data packets which corresponds to a same one of the predetermined time intervals. 
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However, Stanger et al. from the same or similar fields of endeavor disclose a system and 
method for adding auxiliary data to empty space that results from compressing a data stream for 
transmission (see column 2 lines 7-37). Specifically, the system and method partitions video and 
audio data into data packets (see column 5 lines 10-27 and figures 3a-3b), compresses the 
packets to provide empty regions (see column 5 lines 31-65 and figure 4), and inserts auxiliary 
data in the empty spaces (see column 7 lines 41-65 and figure 5). Thus, it would have been 
obvious to the person of ordinaiy skill in the art at the time of the invention to use the system and 
method for adding auxiliary data to empty space that is the result of compressing a data stream as 
taught by Stanger et al. with the method for transmitting and receiving synchronized bio- 
information with audio and video information as taught by Fernandez et al. and Kumar et al. 

As noted above in reference to Kumar et al., the bio-information is synchronized with 
respective audio and video information to form a real-time data stream . Thus, bio-information 
must be multiplexed with audio/video information that all correspond to the same time period. If 
the respective bio-information is not multiplexed with audio/video information that all 
correspond to the same time period, then one of the data information would arrive later. As a 
result, the final stream containing video, audio, and bio-information would be delayed until the 
late data arrives. In this case, the delay would prevent the real-time streaming of the combined 
data streams as disclosed by Kumar et al. Thus, the packetized bio-information of a time period 
must be multiplexed in empty regions that correspond to audio and video packets of the same 
time period to ensure that the bio-information is synchronized with the audio and video 
information to form a real-time combined data stream as disclosed by Kumar et al. The system 
and method for adding auxiliary data to empty space in a data stream can be implemented by 
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inserting the synchronized bio-information as taught by Fernandez et al. and Kumar et al. to the 
empty space that results from compressing the audio and video data using the encoders and 
multiplexers as taught by Stanger et al. The motivation for using the system and method for 
adding auxiliary data to empty space that results from compressing a data stream as taught by 
Stanger et al. with the method for transmitting and receiving synchronized bio-information with 
audio and video information as taught by Fernandez et al. and Kumar et al. is to improve the 
efficiency of the system by creating more payload capacity in the existing transmission channels. 

For claim 51, Fernandez et al. discloses an information reproducing apparatus further 
comprising: means for receiving the multiplexed data stream through a transmission medium 
(see figure 2, which recites a client 20 that uses network interface 24 for receiving the sensed 
data stream). 

For claim 52, Fernandez et al. discloses an information reproducing apparatus further 
comprising: means for reading out the multiplexed data stream from a recording medium (see 
column 3 lines 1-17, which recite a memory implement on a semiconductor for reading out 
sensed signals). 

For claim 53, Fernandez et al. discloses an information reproducing apparatus wherein 
the at least one individual includes a speaker, a player, an actor, or a conductor who serves as a 
source of the audio information, and/or a performer or a photographed person who is included 
within the video information (see column 7 lines 1-9, which recite real-time conference that 
transmits information of the speaker including bio-information gathered from multi- sensor array 
50). 
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For claim 54, Fernandez et al. discloses an information reproducing apparatus wherein 
the at least one individual includes a listener present at the given location when the audio 
information is acquired and/or a viewer present at the given location when the video information 
is acquired (see column 7 lines 1-9, which recite real-time conference that transmits information 
of the listener including bio-information gathered from multi-sensor array 50). 

For claim 55, Fernandez et al. disclose a recording medium having recorded (see column 
2 lines 4-14, which recite storing audio, video, and bio-information for transmission) therein: 

a multiplexed data stream comprised of respective pluralities of first data packets of one 
or more of audio information and video information and corresponding respective pluralities of 
second data packets of statistically processed bio-information (see figure 2 and column 2 lines 
23-34, which recites client 20 including network interface 24 used to receive data streams 
including audio, video, and bio-information that are decomposed and respectively distributed to 
audio device 46, display 32, and mechanical device 48), 

the multiplexed data stream having each respective plurality of second data packets of the 
statistically processed bio-information (see column 4 lines 8-19 and column 7 lines 1-9, which 
recite generating digital packets comprising the collected and interpreted bio-information 
signals collected by biometric sensor array module 50 for real-time exchange of live audio/video 
and the collected bio-information between clients), 

the one or more of audio information and video information being acquired at a 
performance at a given location (see figure 2 and column 2 lines 23-34, which recite a 
microphone 44 and camera 42 for acquiring audio and visual information), 
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the bio-information being of at least one individual present at the given location and 
being detected concurrent with the acquiring of the one or more of audio information and video 
information (see figure 2, figure 4, column 2 lines 23-34, and column 7 lines 1-9, which recite a 
biometric sensor array module 50 that detects bio-information used for real-time conference that 
concurrently transmits audio/video information as well as bio-information gathered from multi- 
sensor array 50), 

the statistically processed bio-information being generated by performing statistical 
processing of the detected bio-information over predetermined time intervals corresponding to 
respective pluralities of the first data packets (see column 3 lines 48-67 and column 4 lines 1-19, 
which recite statistical processing of the bio-information by collecting and interpreting the mix- 
mode signals of the bio sensors 50 into findings, flags, warnings, or other indications provided 
as feedback). 

Fernandez et al. disclose all the subject of the claimed invention with the exception 
wherein the statistical processing of the bio-information is specifically performed over 
predetermined time intervals corresponding to respective pluralities of the first data packets of 
the one or more of audio information and video information, wherein the packetizing of 
statistically processed bio-information is specifically packetized into respective pluralities of 
second data packets corresponding to the respective pluralities of the first data packets. 

However, Kumar et al. disclose a system and method for real-time retrieval, assessment, 
analysis, and transmission of physiological data such as EKG, EMG, EEG, blood pressure, body 
temperature, and pulse measurements (see column 5 lines 47-61 and column 6 lines 50-56). The 
real-time streaming of such bio-information is augmented by transmission of audio and video 
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streams that are synchronized with the respective bio-information (see column 7 lines 4-14). 
Thus, it would have been obvious to the person of ordinary skill in the art at the time of the 
invention to use the system and method for communicating bio-information and respectively 
synchronized audio and video information as taught by Kumar et al. with the method for 
transmitting and receiving bio-information with audio and video information as taught by 
Fernandez et al. The packetized bio-information must be multiplexed and transmitted such that 
it is temporally adjacent to the audio and video packets to ensure that the bio-information is 
synchronized with the audio and video information to form a real-time combined data stream. If 
the bio-information and audio/video information of a corresponding time period are not 
multiplexed together, then one of the data information would arrive later. As a result, the final 
stream containing video, audio, and bio-information would be delayed until the late data arrives. 
In this case, the delay would prevent the real-time streaming of the combined data streams as 
disclosed by Kumar et al. Thus, the packetized bio-information must be multiplexed together 
with the audio and video packets to ensure that the bio-information is synchronized with the 
audio and video information to form a real-time combined data stream as disclosed by Kumar et 
al. The motivation for using the system and method for transmitting bio-information and 
respectively synchronized audio and video information as taught by Kumar et al. with the 
method for transmitting bio-information with audio and video information as taught by 
Fernandez et al. is to enhance the communication experienced by users using transmission of 
audio/video information with respectively synchronized bio-information. 

Fernandez et al. and Kumar et al. disclose all the subject of the claimed invention with 
the exception of specifically partitioning the acquired one or more of audio information and 
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video information into lengths of partitioned data each of which corresponds to a predefined time 
interval, compressing and packetizing the lengths of partitioned data into a stream of first data 
packets having empty regions resulting from the compression of lengths of the partitioned data, 
and multiplexing the stream of first data packets with the second data packets by inserting a 
respective plurality of second data packets, which corresponds to one of the predetermined time 
intervals, into one of the empty regions of the stream that is adjacent to a respective plurality of 
first data packets which corresponds to a same one of the predetermined time intervals. 

However, Stanger et al. from the same or similar fields of endeavor disclose a system and 
method for adding auxiliary data to empty space that results from compressing a data stream for 
transmission (see column 2 lines 7-37). Specifically, the system and method partitions video and 
audio data into data packets (see column 5 lines 10-27 and figures 3a-3b), compresses the 
packets to provide empty regions (see column 5 lines 31-65 and figure 4), and inserts auxiliary 
data in the empty spaces (see column 7 lines 41-65 and figure 5). Thus, it would have been 
obvious to the person of ordinary skill in the art at the time of the invention to use the system and 
method for adding auxiliary data to empty space that is the result of compressing a data stream as 
taught by Stanger et al. with the method for transmitting and receiving synchronized bio- 
information with audio and video information as taught by Fernandez et al. and Kumar et al. 

As noted above in reference to Kumar et al., the bio-information is synchronized with 
respective audio and video information to form a real-time data stream . Thus, bio-information 
must be multiplexed with audio/video information that all correspond to the same time period. If 
the respective bio-information is not multiplexed with audio/video information that all 
correspond to the same time period, then one of the data information would arrive later. As a 
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result, the final stream containing video, audio, and bio-information would be delayed until the 
late data arrives. In this case, the delay would prevent the real-time streaming of the combined 
data streams as disclosed by Kumar et al. Thus, the packetized bio-information of a time period 
must be multiplexed in empty regions that correspond to audio and video packets of the same 
time period to ensure that the bio-information is synchronized with the audio and video 
information to form a real-time combined data stream as disclosed by Kumar et al. The system 
and method for adding auxiliary data to empty space in a data stream can be implemented by 
inserting the synchronized bio-information as taught by Fernandez et al. and Kumar et al. to the 
empty space that results from compressing the audio and video data using the encoders and 
multiplexers as taught by Stanger et al. The motivation for using the system and method for 
adding auxiliary data to empty space that results from compressing a data stream as taught by 
Stanger et al. with the method for transmitting and receiving synchronized bio-information with 
audio and video information as taught by Fernandez et al. and Kumar et al. is to improve the 
efficiency of the system by creating more payload capacity in the existing transmission channels. 

For claim 56, Fernandez et al. discloses a recording medium having recorded wherein 
least one individual includes a speaker, a player, an actor, or a conductor who serves as a source 
of the audio information, and/or a performer or a photographed person who is included within 
the video information (see column 7 lines 1-9, which recite real-time conference that transmits 
information of the speaker including bio-information gathered from multi-sensor array 50). 

For claim 57, Fernandez et al. discloses a recording medium having recorded wherein 
the at least one individual includes a listener present at the given location when the audio 
information is acquired and/or a viewer present at the given location place when the video 
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information is acquired (see column 7 lines 1-9, which recite real-time conference that transmits 
information of the listener including bio-information gathered from multi-sensor array 50). 

For claim 59, Fernandez et al. discloses a recording medium having recorded wherein 
the bio-information is selected from the group consisting at least one of body motion, 
myoelectricity, body surface temperature, skin sweating, skin resistance, pulse, breath, micro- 
vibration, cardioelectricity, heartbeat, and blood pressure (see column 4 lines 20-29, which recite 
detecting temperature, pressure, physiological vital information such as pulse and blood 
pressure using the multi-sensor module 50). 



Response to Arguments 

10. It is noted with appreciation that the Applicants have carefully considered the prior art 
rejections of the pending claims. Applicant's arguments with respect to claims 1-3, 7-11, 15-19, 
23-28, 32-57 and 59 have been considered but are moot in view of the new ground(s) of 
rejection. Although new grounds of rejection have been introduced, the Examiner's response to 
the Applicants' arguments regarding the newly added limitations is provided for the Applicants' 
consideration. As noted above in reference to Kumar et al., the bio-information is synchronized 
with respective audio and video information to form a real-time data stream . Thus, bio- 
information and audio/video information that all correspond to the same time period must be 
multiplexed together. If the respective bio-information is not multiplexed with audio/video 
information that all correspond to the same time period, then one of the bio-information, audio, 
or video data that correspond to the same time period will arrive late. As a result, the combined 
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stream containing video, audio, and bio-information would be delayed until the late data arrives. 
In this case, the delay would prevent the real-time streaming of the combined data streams as 
disclosed by Kumar et al. Thus, the packetized bio-information of a time period must be 
multiplexed in empty regions that correspond to audio and video packets of the same time period 
to ensure that the bio-information is synchronized with the audio and video information to form a 
real-time combined data stream as disclosed by Kumar et al. 



Conclusion 

1 1 . Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to BEN H. LIU whose telephone number is (571)270-31 18. The 
examiner can normally be reached on 9:00AM to 6:30PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Ricky Ngo can be reached on (571)272-3139. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 
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like assistance from a USPTO Customer Service Representative or access to the automated 
information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



/Ricky Ngo/ 

Supervisory Patent Examiner, Art Unit 
2464 



